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•  Organizing  and  Training  for  SDT&E 

•  Test  Approaches 

•  AFFTC  Application  Examples 

•  Way  Forward 
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Organizing  and  Training  for  SDT&E 


AFFTC 


•  AFFTC  approach  is  to  employ  a  shop  of  statistical  experts  (a 
stable  of  PhDs)  to  provide  assistance  to  engineers  during 
test  planning  and  data  analysis 

-  Difficult  to  turn  discipline  engineers  into  statistics  experts 
(nor  do  we  want  to) 


-  Many  tests  are  unique,  so  “cookbook”  planning 
techniques  are  usually  not  possible 

•  Engineers  and  managers  receive  several  courses  on 

statistics  and  DOE  so  they  can  communicate  with  the  experts 


*  Use  a  variety  of  tools:  R,  JMP,  Matlab,  etc 


•  Monthly  information  exchange  meeting  where  technical 
experts  describe  use  of  statistics  in  their  discipline: 


Flight  controls  Modeling  &  Simulation 

Collision  avoidance  Guidance,  Nav,  ID 

Performance  Communications 

Human  Systems  Integration  C4ISR 

Radar  Systems  Weapons  Systems 

Electronic  Attack  Anechoic  Facility 


R&M 

Avionics 

Threat  Signals 

Propulsion 

Structures 

Subsystems 
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Test  Approaches 
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AFFTC 


•  Statistical 

-  Designed  Experiments 

-  Observational  Studies  (may  have  limited  scope  of 
inference  based  on  test  design) 

•  Demonstrations 

•  SME/Opinion-Based 


•  Primary  difference  is  in  the  outcome  of  each 
approach: 

•  Validity  of  the  result,  and 

*  Scope  of  inference  that  can  be  made  for  that  test 


AFFTC  Application: 

Target  Detection  and  Recognition 


mw 


Test  Objective: 

•  Demonstrate  the  ranges  at  which  a  pilot 
can  recognize  and  identify  certain  ground 
targets  using  imaging  sensors  in  a 
targeting  pod 

Test  Approach: 

•  DOE  with  four  factors  and  four  response 
variables 

•  Augmented  flight  tests  with  post-flight 
video  review 


Analysis  Approach: 


Paired  t-test  to  compare  in-flight  data  to 
post-flight  data 

ANOVA  to  determine  significant  factors, 
recognition  ranges,  and  identification 
ranges 


No  statistical  difference,  but  close! 


Results: 


Variance  Components 


Recognition  Range 


Target'lmage  Processing;  Weighted  Means 
Vertical  bars  denote  0.95  confidence  intervals 
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Tank  Humvee  Crane  Fire  Truck 

Target 


AFFTC  Application: 

Endurance  Verification 


/me 


Test  Objective: 

•  Determine  if  an  aircraft,  in  mission 
capable  configuration,  has  a  minimum 
total  endurance  of  X  hours  plus 
appropriate  fuel  reserves  IAW  AF 
instructions 


Test  Approach: 

•  Collect  flight  data  regarding  drag  and  lift 
coefficients,  fuel  flow,  and  static  pressure 
error 


Analysis  Approach: 

•  Estimate  uncertainties  associated  with 
weight,  outside  air  temperature,  thrust, 
fuel,  flow,  drag  coefficient,  and 
calibrated  airspeed 

•  Use  Monte-Carlo  analysis  to  generate 
expected  range  and  endurance 


Results: 


Histogram  with  Normal  Distribution  Curve  for  912  Runs 
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Way  Ahead 


AFFTC 


•  Developing  handbook  to  include  overview  of 
statistics  and  AFFTC  best  practices 

•  Adding  statistics  input  and  review  to  our  test 
planning  and  reporting  processes 

•  Planning  pathfinder  projects  in  each  technical 
discipline 

•  Potential  research  topics: 

-  New  flight  test  techniques 

-  Analysis  of  time  series  data 

-  Quantifying  measurement  uncertainty 

-  Bayesian  techniques 
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